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Abstract 

The measurement rate of cameras that take spatially 
multiplexed measurements by using spatial light mod¬ 
ulators (SLM) is often limited by the switching speed 
of the SLMs. This is especially true for single-pixel 
cameras where the photodetector operates at a rate 
that is many orders-of-magnitude greater than the SLM. 
We study the factors that determine the measurement 
rate for such spatial multiplexing cameras (SMC) and 
show that increasing the number of pixels in the de¬ 
vice improves the measurement rate, but there is an opti¬ 
mum number of pixels (typically, few thousands) beyond 
which the measurement rate does not increase. This mo¬ 
tivates the design of LiSens, a novel imaging architec¬ 
ture, that replaces the photodetector in the single-pixel 
camera with a ID linear array or a line-sensor. We illus¬ 
trate the optical architecture underlying LiSens, build a 
prototype, and demonstrate results of a range of indoor 
and outdoor scenes. LiSens delivers on the promise of 
SMCs: imaging at a megapixel resolution, at video rate, 
using an inexpensive low-re solution sensor. 


1. Introduction 

Many computational cameras achieve novel capabil¬ 
ities by using spatial light modulators (SLMs). Exam¬ 
ples include light field cameras [16,24], lensless cam¬ 
eras [9], and compressive cameras [4,7, 13,20]. The 
measurement rate of these cameras is often limited by 
the switching speed of the SLM used. This is especially 
true for the compressive single-pixel camera (SPC) [4] 
where a photodetector is super-resolved by a digital mi¬ 
cromirror device (DMD). The SPC inherits both the spa¬ 
tial resolution of the DMD as well as its operating rate. 
On one hand, this allows capturing images at a high res¬ 
olution in spite of the sensor being a single pixel. This is 
especially beneficial while imaging in non-visible wave¬ 
bands, where the combination of a single photodetector 
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Figure 1. Motivation. Varying the resolution of the sensor 
in a multiplexing camera provides interesting tradeoffs. A 
single-pixel camera is inexpensive but has limited measure¬ 
ment rate. Full-frame sensors deliver high measurement rates 
but often at a steep price especially in non-visible wavebands. 
The proposed camera, LiSens, uses a 1024 pixel line-sensor 
and achieves a measurement rate in MHz. (right) A 1024 x 768 
pixel image of an outdoor scene using our prototype. 


coupled with a high-resolution DMD often provides an 
inexpensive alternative to full-frame sensors.^ On the 
other hand, the SPC also inherits the operating rate of 
the DMD which, for commercially-available units, is in 
tens of kHz. At this measurement rate, acquiring im¬ 
ages and videos at the spatial resolution of the DMD is 
feasible only at a very low temporal resolution. 

It is instructional to compare the two widely differing 
models of sensing (see Figure 1) in terms of their cost, 
determined by the number of pixels in the camera, and 
measurement rate, defined as the number of measure¬ 
ments that the device obtains in unit-time. Full-frame 
2D sensors that rely on Nyquist sampling are capable 
of achieving measurement rates in tens of MHz, but are 
prohibitively expensive in many non-visible wavebands. 
The SPC, which uses multiplexing as opposed to sam¬ 
pling, has low measurement rates, but is inexpensive for 
exotic wavebands. However, these imaging models are 
only the two extremes of a continuous camera design 

^Megapixel sensors in short-wave infrared, typically constructed 
using InGaAs, cost more than USD 100k. 








space with varying number of pixels. Our main observa¬ 
tion is that the design that achieves the maximum mea¬ 
surement rate at low cost lies between the two extremes. 

Based on this observation, we propose a novel archi¬ 
tecture for video compressive sensing (CS) that is capa¬ 
ble of delivering measurement rates in MHz. The pro¬ 
posed camera, which we call LIne-SENSor-based com¬ 
pressive camera (LiSens), uses a ID array of pixels to 
observe a scene via a DMD. Each pixel on the sensor is 
optically mapped to a row of micromirrors on the DMD 
and hence, the camera obtains a coded line-integrals of 
the image formed on the DMD. The high measurement 
rate of LiSens enables CS of images and videos at high 
spatial resolutions and reasonable frame-rates. We pro¬ 
vide the optical schematic for building such a camera 
and illustrate its novel capabilities with a lab prototype. 

2. Prior work 

Multiplexed imaging. This paper relies heavily on the 
concept of optical multiplexing. Suppose that we seek to 
sense a signal x G A conventional camera samples 
this signal directly and has an image formation model 
given by 

y = x + e, 

where e is the measurement noise. Multiplexing ex¬ 
pands the scope of the camera by allowing it to obtain 
any arbitrary linear transformation of the signal: 

y = Ax -F- e. 

Assuming that A G is invertible, the signal can 

be estimated as x = A~^y. It is well known that, when 
the entries of A = [aij] are restricted to \aij\ < 1, using 
a Hadamard measurement matrix is optimal provided 
the noise is signal independent [6]. Note that in the ab¬ 
sence of any priors, we need at least N measurements to 
recover an A^-dimensional signal. 

Compressive sensing (CS). CS aims to sense a signal 
X G from an under-determined linear system, i.e., 
measurement of the form 

y = <|)X + e, 

where G with M < N. For an arbitrary signal 

in R^ this is impossible since the map <l> : R^ i-G R^ is 
many-to-one and non-invertible. CS handles this by re¬ 
stricting the signal x to belong to a distinguished class; 
for example, sparse signals in a transform basis. The 
main results of CS state that when the measurement ma¬ 
trix <I> has a special structure and x is AT-sparse in a 
transform basis, then we can robustly recover x, pro¬ 
vided M = 0{Klog{N/K)) [1]. 


Video models for CS. There have been many mod¬ 
els proposed to handle CS of videos. Wakin et al. [26] 
proposed the use of 3D wavelets as a sparsifying trans¬ 
form for videos. This model is further refined in Park 
and Wakin [18] where a lifting scheme is used to tune 
a wavelet to better approximate the temporal variations. 
Dictionary-based models were used in Hitomi et al. [7] 
for temporal resolution of videos; it is shown that a 
highly overcomplete dictionary provides high quality re¬ 
constructions. Inspired by the use of motion-flow mod¬ 
els in video compression, Reddy et al. [20] and subse¬ 
quently, Sankaranarayanan et al. [21] employed an iter¬ 
ative strategy where optical flow derived from an initial 
estimate is used to further constrain the video recovery 
problem. This provides a significant improvement in re¬ 
construction quality, albeit at a high computational cost. 
Gaussian mixture models were used in [27] for video 
CS; a hallmark of this approach is that the mixture model 
is learned directly from the compressive measurements 
providing a model that is tailored to the specifics of the 
scene being reconstructed. 

Compressive imaging hardware prototypes. The 

original prototype for the SPC used a DMD as the pro¬ 
grammable SLM [4]. In [9], a variant of the SPC is pro¬ 
posed where an LCD panel is used in place of the DMD; 
the use of a transmissive light modulator enables a lens¬ 
less architecture. Sen and Darabi [23] use a camera- 
projector system to construct an SPC exploiting a con¬ 
cept called dual photography [22]; the hallmark of this 
system is its use of active illumination. 

There have been many multi-pixel extensions to the 
SPC. The simplest approach [11] is to map the DMD to a 
low-resolution sensor array, as opposed to a single pho¬ 
todetector, such that each pixel on the sensor observes 
a non-overlapping “patch” or a block of micromirrors 
on the DMD.^ SMCs based on this design have been 
proposed for sensing in mid-wave infrared camera [15] 
and short-wave infrared [3]. Measurement matrix de¬ 
signs for such block-based compressive imaging archi¬ 
tectures are presented in Kerviche et al. [12] and Ke and 
Lam [10]. 

While we focus on spatial multiplexing, many archi¬ 
tectures have been proposed for multiplexing other at¬ 
tributes of a scene including temporal [5, 7, 8,14, 20], 
angular [16,24], and spectral [13,25]. All of these ar¬ 
chitectures use a high-resolution sensor and sacrifice this 
spatial resolution partially to obtain higher resolution in 

^The interested reader is referred to earlier work by Nayar et al. 
[17] where a scene is observed by a sensor via a DMD to enable pro¬ 
grammable imaging. However, this device was not used for spatial 
multiplexing or compressive sensing. 



time, spectrum and/or angle. In contrast, the imaging 
architecture proposed in this paper takes a device with 
a high temporal resolution and sacrifices the temporal 
resolution in part to obtain higher spatial resolution. 

3. Measurement rate of a camera 

The measurement rate of a camera is given by the 
product of its spatial resolution, in pixels, and the tem¬ 
poral resolution, in frames per second. A spatial multi¬ 
plexing camera (SMC) captures one or more coded lin¬ 
ear measurements of a scene via a spatial light modu¬ 
lator (e.g., DMD). Multiple measurements are taken se¬ 
quentially by changing the code displayed on the DMD. 
Thus, the measurement rate of an SMC is limited by the 
resolution and the frame-rate of the sensor, as well as the 
resolution and switching speed of the DMD.^ The oper¬ 
ating rate of the DMD is determined by the rate at which 
the micromirrors can be switched from one code into an¬ 
other. Denoting this rate as Rdmd Hz, we note that the 
frame rate of the SMC cannot be greater than Rdmd 
since the specific linear mapping from the scene to sen¬ 
sor is determined by the micromirror code. The frame 
rate of a sensor is also limited by its readout speed — 
typically, determined by the operating rate of the analog- 
to-digital converter (ADC) in the readout circuit. Sup¬ 
pose that we have an ADC that can perform readout at 
a rate of Radc Hz. Then, given a frame with F pixels, 
the ADC limits the sensor to Radc IF frames per sec¬ 
ond. Hence, the measurement rate of the SMC is given 
as 


F X min (^Rdmd, 


meas. per sec. 


Operating speeds of commercially-available DMDs 
are 2-3 orders of magnitude lower than that of ADCs 
in sensors. As a consequence, for small values of F, 
the measurement rate of an SMC is dominated by the 
operating speed of the DMD (see Figure 2). Once 
F > Radc I Rdmd^ then the bottleneck shifts from 
the DMD to the ADC. Further, the smallest sensor reso¬ 
lution (in terms of number of pixels) for which the mea¬ 
surement rate is maximized is 


(minimum number of pixels) F^in = Radc / Rdmd • 

In essence, at F = Fmin can obtain the measure¬ 
ment rate of a full-frame sensor but with a device with 
potentially a fraction of the number of pixels. This is 

^Compressive cameras can further increase the spatial or temporal 
resolution by using scene priors. 



Figure 2. Measurement rate and sensor resolution. There 
are two regimes of operations: for F < Fmin, the bottleneck 
is the DMD and for F > Fmin, the bottleneck is the sensor 
readout. At F = Fmin lies a design that obtains the highest 
measurement rate with the least sensor resolution. 


invaluable for sensing in many wavebands, for example 
short-wave infrared. 

As a case study, consider an SMC with a DMD op¬ 
erating rate Rdmd = 10^ Hz and an ADC with an op¬ 
erating rate Radc = 10^ Hz. Then, for a sensor with 
^min = 10^ pixels, we can obtain 10^ measurements 
per second. An SPC, in comparison, would only provide 
10^ measurements per second. This significant increase 
in measurement rate motivates the multi-pixel SMC de¬ 
sign that we propose in the next section. 

4. LiSens 

The optical setup of LiSens is illustrated in Figure 3. 
The DMD splits the optical axis into two axes — one 
each for ±11° orientations of the micromirrors. Along 
one of these axes, the 2D image of the scene formed on 
the DMD plane is mapped onto the ID line-sensor using 
a relay lens and a cylindrical lens. The image captured 
by the line-sensor can be represented as a ID integral 
of the 2D image (along rows or columns) formed on the 
DMD plane. This is achieved by aligning the axis of the 
cylindrical lens with that of the line-sensor and placing 
it so that it optically mirrors the aperture plane of the 
relay lens onto the sensor plane (see Figure 4). 

Suppose that the physical dimensions of the DMD 
and the line-sensor are {wD^hD) and respec¬ 

tively, with Hl ^ wl- The relay lens is selected to 
produce a magnification of wlIwd so that the DMD 
maps to the line-sensor along its width; this determines 
the ratio of the focal lengths used in the relay lens. The 
exact value of the focal lengths is optimized to minimize 
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Figure 3. Schematic of the LiSens imaging architecture. 
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Figure 4. The DMD-to-sensor mapping. Shown are the ray 
diagrams for the mapping of the DMD to line-sensor along two 
orthogonal directions — (top) along and (bottom) orthogonal 
to the axis of cylindrical lens. The cylindrical lens focuses the 
aperture plane of the relay lens onto the sensor plane. Hence, 
the sensor plane is focused on the DMD along one-axis (top) 
but defocused along the other (bottom). 



Figure 5. Field-of-view enhancement by the cylindrical lens. 

Shown are images acquired with our prototype. 



(a) simultaneous exposure and readout (b) sequential exposure and readout 


Figure 6. Benefits of frame transfer. The DMD was operated 
with 400/is per pattern. The sensor had a readout of 350/is per 
frame, (a) Simultaneous readout allows for a 400//S exposure, 
(b) Sequential readout cut the exposure time down to 50/is, 
resulting in a noisy image. Shown are images acquired with 
our prototype. 


throughput, we could obtain flexibility in both the choice 
of the cylindrical lens (focal length) as well as its posi¬ 
tioning. Figure 5 shows the improvement in the vertical 
field-of-view when the cylindrical lens is introduced. 


vignetting and needs to be determined along with the ob¬ 
jective lens. If wd = wl, can use a 1:1 relay lens 
with a focal length and the distance between the relay 
lens and the line-sensor is 2/^. The parameters and the 
placement of the cylindrical lens are determined so that 
the aperture of the relay lens is magnified (or shrunk) 
within the line-sensor. This corresponds to a magnifi¬ 
cation of h 1,1 dr, where is the diameter of the relay 
lens. Let /c be the focal length of the cylindrical lens, 
and suppose that it is placed at a distance of Uc from the 
line-sensor and from the relay lens. Then, 


^. 1 1 1 
tt-c I - 5 

Uc Vc /c Vc 


dr 


Hence, 


fc = Vr 


hr 


dr 


dr -\- h]^ dr -\- flL 


2fr 


( 1 ) 


In practice, we observe that, for a marginal loss of light 


4.1. Practical advantages 

Before we proceed, it is worth pondering on alter¬ 
native and potentially simpler designs for multi-pixel 
SMCs. An intuitive multi-pixel extension of the SPC 
would be using a low-resolution 2D sensor array [3,15]. 
Having a 2D sensor array provides a simpler mapping 
from the DMD that can be achieved using just relay 
lenses. Yet, there are important considerations that make 
the LiSens camera a powerful alternative to using 2D 
sensors. 

Specifically, it is simpler and inexpensive to obtain 
frame transfer'^ on a line sensor without requiring com¬ 
plex circuitry or loss of light due to reduced fill-factor. 

^Frame transfer is a technology used to enable simultaneous expo¬ 
sure and readout in a sensor. In a traditional camera, during the readout 
process, the sensor is not exposed; this reduces the duty cycle of the 
sensor and leads to inefficiencies in light collection. With frame trans¬ 
fer, there is a separate array of storage pixels beside the photosensitive 
array. After exposure, charge from the pixels are immediately trans¬ 
ferred to the storage array allowing the sensor to be exposed again. 

























Figure 6 demonstrates the benefits of frame transfer on 
our lab prototype. The ID profile of the sensor also pro¬ 
vides the possibility of having a per-pixel ADC; this pro¬ 
vides a dramatic increase in the readout rate of the sen¬ 
sor. Finally, we benefit from the fact that line-sensor 
have long been manufactured for spectroscopy which 
requires very precise, low-noise sensors with high dy¬ 
namic range and broad spectral response; these proper¬ 
ties make them highly desirable for our application. 

4.2. Imaging model 

Suppose the DMD has a resolution of x mi¬ 
cromirrors, and the line-sensor has N pixels such that 
each pixel maps to a row of micromirrors on the DMD. 
At time instant t, let G be the scene image 

formed on the DMD, and let G be the binary 

pattern displayed on the DMD. Then, the measurement 
obtained at the line-sensor, yt G R^, is given as 

yt = (Xt o<Ft)l + et, 

where o denotes the entry-wise product, 1 is the vector 
of ones and is the measurement noise. 

For simplicity, we use DMD patterns where every 
column is the same, i.e., patterns of the form 

where (pt G R^ is a binary vector. This alleviates 
the need for extensive calibration. The sensor measure¬ 
ments are then given as 

yt = Xt^t + et. 

For the experiments in the paper, we use rows of a 
column-permuted Hadamard matrix for 

4.3. Recovery 

Sensing images. When the scene is static, i.e., 
Xt = X for t G {1,..., T}, we can write the imaging 
model as 

Y = [yi,...,yr]=X$ + E, 

where = [0i,..., is the measurement matrix and 
E = [ei,..., Gt] is the measurement noise. 

If T > N and ^ were well-conditioned, say for ex¬ 
ample, a Hadamard matrix, then we can obtain an esti¬ 
mate of the scene image as 

X = (2) 

In practice, it is unreasonable to expect a scene to be 
static over a large duration and hence, we can expect 


T < N. To regularize the inverse problem, we enforce 
sparsity in the gradients of the recovered image using 
a minimum total-variation prior [2]. This leads to the 
following optimization problem. 

rmnTy(X), s.t. ||Y - X<F||i. < e, (3) 

where TV (X) is the total-variation norm that captures 
strength of the image gradients. We used the isotropic 
TV-norm defined as 

N N 

Ty(X) = 

i = l j = l 

where Gx^Gy G R^^^ are the spatial gradients of X. 

Sensing videos. We enforce sparse spatio-temporal 
gradients when recovering time-varying scenes. Specif¬ 
ically, suppose that we obtain measurements {y^, 1 < 
t < T} for a time-varying scene {X^, 1 < t < T}. It is 
often an overkill to recover an image for every frame of 
the sensor since the problem becomes computationally 
overwhelming. Instead, we decide on a target frame-rate 
(say 10 frames per second) and group together succes¬ 
sive measurements so as to obtain the desired frame rate. 
Let {X/c, k = 1,..., Q} be the frames associated with 
the scene at the desired frame-rate. Then, the imaging 
model reduces to 

T 

t = k-Q, [yt,..-,y(+T] = Xk[4>t,---,4>t+^] + Ek 

Grouping together measurements and associating them 
to a frame of a video also reduces the number of pa¬ 
rameters to be estimated. Interested readers are referred 
to [19] for a frequency domain analysis on judicious se¬ 
lection of frame-rate for the SPC. Finally, similar to (3), 
we solve for the video by minimizing a spatio-temporal 
TV-norm constrained by the measurements. 

5. Hardware prototype 

Our proof-of-concept prototype was built using a 
Nikkor 50mm F/1.8 objective lens, a DLP7000 DMD, 
and a Hamamatsu SI 1156-2048-01 line-sensor. The 
DMD had a spatial resolution of 1024 x 768 with a 
micromirror pitch of 13.8//m and a maximum operat¬ 
ing speed of 20 kHz. The line sensor had 2048 pixels 
with a pixel size of 14//m x 1mm; we were only able 
to use 1024 pixels due to our inability to find a cylin¬ 
drical lens that was sufficiently long to span the entire 
line-sensor. Due to the pixel pitch and micromirror pitch 
being nearly the same, we used a 1:1 relay lens and ori¬ 
ented the line sensor so that the line sensor was aligned 






to the width of the DMD. Hence, each pixel on the line 
sensor summed up 768 micromirrors on the DMD. 

Our line sensor provided access to simultaneous ex¬ 
posure and readout using frame transfer. We operated 
the prototype with an exposure of approximately 500/i5. 
Due to buffer limitations in the readout circuit (a Hama¬ 
matsu Cl 1165-01 driver), we were able to obtain 100 
frames at 500/i5 per frame, followed by a cool-down 
time of 60ms. As a consequence, the device provided 
900 frames per second and hence, a measurement rate 
of 9 X 10^ measurements per second. 

For comparison and validation, we built an SPC on 
the off-axis of the DMD (see Figure 7). The measure¬ 
ment rate of the SPC was 20 kHz, the speed of the DMD. 

Figure 7 shows both the modified schematic of the 
camera as well as the optical layout. The subtle differ¬ 
ences to the schematic shown in Figure 3 stem from a 
couple of practical constraints. Recall that the DMD re- 
fiects the optical axis by 22°. The flange distance of 
objective lenses (F-mount and C-mount) is insufficient 
to provide clearance for the cone of light reflected from 
the DMD. To alleviate this, we optically mirror the im¬ 
age plane of the objective lens using a 1:1 relay lens, 
thereby providing ample space for the reflected cone of 
light. We used a 100mm: 100mm achromatic doublets 
for the relay lens, thereby providing an fr = 50mm and 
a cylindrical lens with a focal length of 12.7mm.^ 

Alignment. Misalignment of the optical components, 
in addition to introducing blur, results in a DMD column 
mapping to multiple pixels on the line detector. In our 
prototype, we minimized both effects by manually ad¬ 
justing each component while observing and quantify¬ 
ing sharpness of the images in real-time. While this pro¬ 
duces acceptable results, a calibration procedure is in¬ 
deed required especially to resolve the image especially 
at the boundaries of the field of view. 

Vignetting. The use of relay lenses to extend the op¬ 
tical axis causes significant vignetting. To alleviate this, 
we introduced a field lens at the DMD which dramati¬ 
cally reduces vignetting. A second field lens at the im¬ 
age plane of the objective lens further reduces vignetting 
but with an increase in spatial blur. The results in this pa¬ 
per were produced with a single field lens at the DMD. 

6. Experiments 

We use two metrics to characterize the operating sce¬ 
narios for our experiments: 

^We used relay lenses with diameter dr = 25mm. The value 
suggested in (1) is /c = 4mm. However, to avoid custom designed 
optics, we went with a larger focal length. 
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Figure 7. Prototype of the LiSens camera. 


(i) The under-sampling given as the ratio of the dimen¬ 
sionality of the acquired image to the number of mea¬ 
surements, i.e., if we seek to acquire an Ni x N 2 im¬ 
age with M measurements, then the under-sampling is 
N 1 N 2 /M. When the under-sampling is 1, the system 
is invertible and we use (2) to recover the image. For 
values greater than 1, we use the TV prior and solve (3). 

(ii) Temporal resolution / capture duration per frame. 
Recall, from Section 4.3, that we pool together mea¬ 
surements and associate them with a single recovered 
frame. This determines the capture duration per frame 
and its reciprocal, the temporal resolution. The smaller 
the capture duration, the fewer the measurements that 
we acquire and hence, the greater the under-sampling. 

These two metrics are linked by the measurement rate 
of the camera and the resolution at which we operate the 
DMD. For a capture duration of r seconds, the under¬ 
sampling is given as 

{NiN2)/{t X measurement rate) 

where the DMD resolution is Ni x N 2 . Recall, that the 
measurement rate of our LiSens prototype is nearly 1 
MHz while that of the SPC is 20 kHz. 

Finally, we map the ±1 entries of the Hadamard 
matrix to 0/1 by mapping the ‘-I’s to ‘O’s; in post¬ 
processing, the measurements corresponding to the ±1 
matrix are obtained simply by subtracting the mean 
scene intensity. For time-varying scenes, we repeat the 
all-ones measurement in the Hadamard matrix once ev¬ 
ery 100 measurements to track the mean intensity value. 

Gallery. We present the images recovered by the 
LiSens camera on two indoor static scenes for various 
capture durations in Figure 8. We also quantize the loss 
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Figure 8. LiSens reconstructions for a few scenes under varying capture durations / under-sampling. The reconstructions 
suffer little loss in quality until a capture time of llOms; this corresponds to an under-sampling of 8x. Inset on each figure is the 
reconstruction SNR, in dB, using the 1 x reconstruction as the ground truth. 


in performance with increased under-sampling using the 
reconstruction SNR. Specifically, given a ground truth 
image and a reconstructed image x, the reconstruc¬ 
tion SNR in dB is given as 


-201ogio 


/ ||Xg^ -x|| 2 ~\ 

V l|Xg.||2 )' 


We used the Nyquist rate reconstruction, i.e., under¬ 
sampling of 1X, as the ground truth image. We observe 
that there is little loss in performance until a capture 
duration of 220ms (under-sampling of 4x) and with a 
small drop in performance at 110ms (8x). As we will 
see later, using a spatio-temporal prior improves the re¬ 
construction quality even at an under-sampling of 8 x. 

Comparison to SPC. We compare the performance of 
LiSens and an SPC built on the off-axis of the DMD. 
We image a static scene using both cameras and com¬ 
pare the reconstructed image for varying capture dura¬ 
tions (see Figure 9). Recall that the measurement rate 
of the SPC is limited to 20 kHz while the LiSens has 
nearly 1 MHz. This 50 x gap enables the LiSens camera 
to sense images at a resolution of 1024 x 768 at 5 — 10 
frames per second. As we will see next, using simple 
spatio-temporal priors like a 3D TV norm will enable 
higher-quality reconstructions at an increased temporal 
resolution. We note that the photodetector in the SPC 
had a lower dynamic range than the line-sensor. For a 
fairer comparison, we took multiple runs with the SPC 
and averaged the measurements to artificially boost its 
dynamic range. 


Figure 9 also indicates that LiSens operating at 8x 
under-sampling produces significantly higher quality re¬ 
sults than the SPC operating at 7.4 x under-sampling. 
This is a consequence of the sparsity of images scaling 
sub-linearly with image resolution. That is, if we looked 
at sparsity of images in a wavelet basis as a function 
of image resolution, the ratio of sparsity level to image 
resolution decreases monotonically. This can also be 
attributed to images having a lot of energy in low-freq 
components. Hence, if we attempt to recover a scene 
at 128 X 128 and 1024 x 1024, each at say 8x under¬ 
sampling, then the quality would be significantly better 
for the high-resolution reconstruction. 

Dynamic scenes. Next, we sensed a time-varying 
scene — a bunny on a turntable spinning at one revo¬ 
lution every 15 seconds. We fix a capture duration of 
llOms per frame or equivalently a temporal-resolution 
of 9 frames per second, and recover the frames of the 
video under a 3D total variation prior. Due to high levels 
of measurement noise, the recovered videos were sub¬ 
ject to a 3 x 3 x 3 median-filter. As seen in Figure 10, 
the recovered video preserves fine spatial detail. This 
can be attributed to the use of video priors, which cap¬ 
ture inter-frame redundancies, as well as the high mea¬ 
surement rate enabled to our design. 

This validates the potential of the LiSens architecture 
to acquire scenes at high spatial resolution (1024 x 768) 
and with a reasonable temporal resolution. 

Finally, Figure 11 shows frames of a video of an out¬ 
door scene reconstructed with the same parameters as 





















Figure 9. Performance of LiSens and SPC. We look at recovered images for various capture durations and, in the case of the SPC, 
various spatial resolution as well. The remarkable gap in the performance of the devices can be attributed to the difference in their 
measurement rates. Inset in orange are the under-sampling factors at which images were reconstructed. 



Figure 10. Video CS using LiSens. We image a bunny on a turntable (left) and recover the video at a capture duration of 110ms 
per frame (under-sampling of 8x) using a 3D TV prior. Shown are a few recovered frames and spatio-temporal slices. 


the indoor bunny scene. The data was significantly nois¬ 
ier which contributed to the reduced reconstruction qual¬ 
ity. The use of stronger priors such as motion flow mod¬ 
els as in [20,21] could help in such scenarios. 

7. Discussion 

We present a multiplexing camera that is capable of 
delivering very high measurement rates, comparable to 
that of a full-frame sensor, but with a number of pixels 
that is a small fraction. At its core, the proposed camera 
moves the bottleneck in the frame rate of the device from 
the spatial light modulator to the readout of the sensor. 


The ensuing boost in measurement rate enables sensing 
of scenes at video rate and at the full resolution of the 
light modulator. Finally, while our prototype was built 
for visible wavebands, the underlying principles transfer 
mutatis mutandis to other wavebands of interest. 
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Figure 11. Sensing outdoors with the LiSens prototype. 
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